Auto-cleaning system, cleaning robot and method of controlling the cleaning robot

ABSTRACT

A cleaning robot that performs cleaning while travelling a space to be cleaned, the cleaning robot including: a travelling unit that moves the cleaning robot; a cleaning unit that cleans the space to be cleaned; an image capturing unit that captures an image viewed from the cleaning robot; a voice input unit to which a user&#39;s voice instructions are input; and a controller obtaining the user&#39;s motion instructions through the image capturing unit and determining a restricted area in which entry of the cleaning robot is prohibited and/or a focused cleaning area to be intensely cleaned by the cleaning robot based on the user&#39;s motion instructions or the user&#39;s voice instructions when the user&#39;s voice instructions are input through the voice input unit. The restricted area and the focused cleaning area may be input to the cleaning robot through the user&#39;s voice and motion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. application Ser.No. 14/444,189 filed on Jul. 28, 2014, which is related to and claimspriority benefit of Korean Patent Application No. 10-2013-0089652, filedon Jul. 29, 2013 in the Korean Intellectual Property Office, thedisclosures of which are incorporated herein in their entirety byreference.

BACKGROUND

1. Field

The following description relates to a cleaning robot, and moreparticularly, to a cleaning robot that determines a restricted area anda focused cleaning area of the cleaning robot by detecting a user'smotion.

2. Description of the Related Art

A cleaning robot is a device that travels a space to be cleaned toautomatically clean the space to be cleaned by inhaling foreignsubstances including dust from a floor without user manipulation. Thatis, the cleaning robot cleans the space to be cleaned while travelingthe space to be cleaned.

In the cleaning robot according to the related art, an entry preventiontape is attached to the floor with respect to a restricted area whichthe cleaning robot is prohibited from entering, such as stairs or averanda. However, setting the restricted area of the cleaning robotusing the entry prevention tape in this way causes discomfort and theinconvenience of attaching and re-attaching the entry prevention tape bythe user when the user wants to change the restricted area.

Also, in the cleaning robot according to the related art, when there isan area to be more intensely cleaned or to be first cleaned by the user,there are inconveniences that the cleaning robot should be manuallymanipulated and should be moved to the area to be more intensely cleanedor to be first cleaned and then focused cleaning instructions should begiven to the cleaning robot.

SUMMARY

Additional aspects and/or advantages will be set forth in part in thedescription which follows and, in part, will be apparent from thedescription, or may be learned by practice of the disclosure.

Therefore, it is an aspect of the present disclosure to provide acleaning robot that is capable of determining a restricted area and afocused cleaning area of the cleaning robot by detecting a user's voiceor motion or in which the restricted area and the focused cleaning areamay be easily set using a portable mobile terminal.

Additional aspects of the disclosure will be set forth in part in thedescription which follows and, in part, will be apparent from thedescription, or may be learned by practice of the disclosure.

In accordance with one aspect of the present disclosure, a cleaningrobot that performs cleaning while travelling a space to be cleaned, thecleaning robot includes: a travelling unit that moves the cleaningrobot; a cleaning unit that cleans the space to be cleaned; an imagecapturing unit that captures an image viewed from the cleaning robot; avoice input unit to which a user's voice instructions are input; and acontroller that obtains the user's motion instructions through the imagecapturing unit when the user's voice instructions are input through thevoice input unit and determines a restricted area and/or a focusedcleaning area to based on the user's motion instructions.

The voice input unit may include at least three microphones to which theuser's voice instructions are input.

The controller may estimate a position of a user based on a differencein times at which the user's voice instructions are input to the atleast three microphones.

The controller may control the travelling unit to rotate the cleaningrobot so that the image capturing unit is directed toward the positionof the user.

The image capturing unit may include a camera that captures atwo-dimensional image of the user and an infrared sensor that obtainsdistance information of the user captured by the camera.

The controller may detect a motion of the user based on the image of theuser captured by the image capturing unit and the distance informationof the user obtained by the image capturing unit.

The controller may detect a hand and a shoulder of the user from theimage of the user and may determine coordinates of the hand and theshoulder of the user based on the distance information of the user.

The controller may determine an area instructed by the user based on thecoordinates of the hand and the shoulder of the user and may determinethe area instructed by the user as the restricted area and/or thefocused cleaning area.

In accordance with another aspect of the present disclosure, a method ofcontrolling a cleaning robot that performs cleaning while travellingabout a space to be cleaned, the method includes: when voiceinstructions are input from a user, obtaining an image of the user;obtaining motion of the user based on the image; an determining arestricted area in which entry of the cleaning robot is prohibitedand/or a focused cleaning area to be intensely cleaned by the cleaningrobot based on the user's voice instructions and the image of the user.

The obtaining of the image may include: determining a position of theuser based on a difference in times at which the user's voiceinstructions are input to a plurality of microphones; rotating thecleaning robot toward the position of the user; obtaining an image ofthe user; radiating infrared rays toward the user; and obtainingdistance information of the user based on the infrared rays reflectedfrom the user.

The obtaining of the motion of the user may include: detecting a handand a shoulder of the user from the image of the user; and determiningcoordinates of the hand and the shoulder of the user using distanceinformation of the user.

The determining of the restricted area and/or the focused cleaning areamay include: determining an area instructed by the user based on thecoordinates of the hand and the shoulder of the user; and determiningthe area instructed by the user as the restricted area and/or thefocused cleaning area.

The method may further include: displaying an image of the space to becleaned; and receiving the restricted area and/or the focused cleaningarea from the user with respect to the image of the space to be cleaned.

The method may further include determining coordinates of the focusedcleaning area and/or the restricted area in the space to be cleanedbased on the coordinates of the focused cleaning area and/or therestricted area input with respect to the image of the space to becleaned.

The method may further include displaying an image of the restrictedarea and/or the focused cleaning area over the image of the space to becleaned.

In accordance with still another aspect of the present disclosure, anauto-cleaning system includes: a cleaning robot that performs cleaningwhile travelling about a space to be cleaned; and a portable mobileterminal that controls the cleaning robot remotely, wherein, whenfocused cleaning area input instructions and/or restricted area inputinstructions are input from a user, the cleaning robot captures an imageof the space to be cleaned and transmits the image of the space to becleaned to the portable mobile terminal, and when the image of the spaceto be cleaned is received, the portable mobile terminal displays theimage of the space to be cleaned and receives a focused cleaning areaand/or a restricted area, from the user.

The cleaning robot may include an image capturing unit that captures animage of the space to be cleaned and receives motion instructions of theuser and a voice input unit that receives voice instructions of theuser, and the cleaning robot may determine the restricted area and/orthe focused cleaning area based on the voice instructions input throughthe voice input unit and the motion instructions input through the imagecapturing unit.

When the voice instructions are input through the voice input unit, thecleaning robot may rotate toward the user and may receive the motioninstructions through the image capturing unit.

The portable mobile terminal may include a touch screen panel on whichthe image of the space to be cleaned is displayed and manipulationinstructions are received from the user, and when the focused cleaningarea and/or the restricted area is input through the touch screen panel,the focused cleaning area and/or the restricted area may beoverlappingly displayed on the image of the space to be cleaned throughthe touch screen panel.

The portable mobile terminal may determine absolute coordinates in thespace to be cleaned that correspond to the focused cleaning area and/orthe restricted area.

The portable mobile terminal may determine image coordinates of thefocused cleaning area and/or the restricted area from the touch screenpanel, may determine relative coordinates of the focused cleaning areaand/or the restricted area in which the cleaning robot is set as anorigin, based on the image coordinates, and may determine absolutecoordinates of the focused cleaning area and/or the restricted area inthe space to be cleaned, based on the relative coordinates and aposition of the cleaning robot.

In the cleaning robot in accordance with one aspect of the presentdisclosure, the cleaning robot may receive the restricted area and thefocused cleaning area through voice and motion of the user.

In the cleaning robot in accordance with another aspect of the presentdisclosure, the cleaning robot may receive the restricted area and thefocused cleaning area only by simple manipulation through the portablemobile terminal of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects of the disclosure will become apparent andmore readily appreciated from the following description of embodiments,taken in conjunction with the accompanying drawings of which:

FIG. 1 schematically illustrates a cleaning robot and a portable mobileterminal in accordance with an embodiment of the present disclosure;

FIG. 2A is a block diagram of control flow of the cleaning robotillustrated in FIG. 1;

FIG. 2B is a block diagram illustrating the configuration of acontroller of the cleaning robot of FIG. 1;

FIG. 3 is a perspective view illustrating the exterior of the cleaningrobot of FIG. 1;

FIG. 4 is a bottom view illustrating the bottom of the cleaning robot ofFIG. 1;

FIG. 5 is a block diagram illustrating control flow of the portablemobile terminal illustrated in FIG. 1;

FIG. 6 is a perspective view illustrating the exterior of the portablemobile terminal of FIG. 1;

FIG. 7 illustrates the case in which the cleaning robot of FIG. 1receives voice instructions from a user;

FIGS. 8A, 8B and 8C illustrate the case in which the cleaning robot ofFIG. 1 determines a user's position through the user's voiceinstructions;

FIG. 9 illustrates the case in which the cleaning robot of FIG. 1obtains an image and voice of a user;

FIGS. 10A and 10B and FIGS. 11A and 11B illustrate the case in which thecleaning robot of FIG. 1 determines coordinates of an area instructed bythe user from the image of the user;

FIG. 12 illustrates the case in which the cleaning robot of FIG. 1transmits the image of the user and an image of a space to be cleanedand information regarding the area instructed by the user to theportable mobile terminal;

FIG. 13 illustrates the case in which the cleaning robot of FIG. 1 andthe portable mobile terminal of FIG. 1 display the image of the space tobe cleaned and the information regarding the area instructed by theuser;

FIG. 14 illustrates the case in which information regarding a focusedcleaning area is input to the portable mobile terminal of FIG. 1;

FIGS. 15 and 16 illustrate the case in which information regarding arestricted area is input to the portable mobile terminal of FIG. 1;

FIG. 17 illustrates a screen on which the portable mobile terminal ofFIG. 1 displays the focused cleaning area and the restricted area;

FIG. 18 is a flowchart illustrating a method of inputting and displayingthe focused cleaning area and the restricted area using the cleaningrobot and the portable mobile terminal of FIG. 1 in accordance with anembodiment of the present disclosure;

FIG. 19 illustrates the case in which the cleaning robot of FIG. 1receives manipulation instructions from the user through the portablemobile terminal;

FIG. 20 illustrates a screen on which information regarding the focusedcleaning area is input through the portable mobile terminal of FIG. 1;

FIGS. 21 and 22 illustrate a screen on which information regarding therestricted area is input through the portable mobile terminal of FIG. 1;

FIG. 23 illustrates a screen on which the portable mobile terminal ofFIG. 1 displays the focused cleaning area and the restricted area; and

FIG. 24 is a flowchart illustrating a method of inputting and displayingthe focused cleaning area and the restricted area using the cleaningrobot and the portable mobile terminal of FIG. 1 in accordance withanother embodiment of the present disclosure.

DETAILED DESCRIPTION

Configurations shown in one or more embodiments and the drawings of thepresent disclosure are merely exemplary embodiments of the discloseddisclosure, and it should be understood that there are various modifiedexamples that may replace embodiments and the drawings of the presentdisclosure at the time of filing of the present application.

Reference will now be made in detail to one or more embodiments of thepresent disclosure, examples of which are illustrated in theaccompanying drawings, wherein like reference numerals refer to likecomponents throughout.

FIG. 1 schematically illustrates a cleaning robot and a portable mobileterminal in accordance with an embodiment of the present disclosure.

A cleaning robot 100 illustrated in FIG. 1 is a device that cleans dustfrom a floor while traveling a space to be cleaned without usermanipulation. Also, the cleaning robot 100 receives manipulationinstructions from a user U such as a user's motion or voice and performscleaning according to the user's manipulation instructions.

In detail, the cleaning robot 100 captures an image including the user Uand the space to be cleaned using an image capturing unit, such as acamera or video camera, and analyzes a user's motion included in thecaptured image so as to detect manipulation instructions to beinstructed by the user U. The image may include other objects such as asofa S and a table T. When the manipulation instructions of the user Uare detected, the cleaning robot 100 cleans the space to be cleanedaccording to the detected manipulation instructions. Also, the cleaningrobot 100 transmits the captured image and the detected manipulationinstructions to a portable mobile terminal 200 of the user U so that theuser U may check an operation of the cleaning robot 100. In addition,the user U may check the operation of the cleaning robot 100 through theportable mobile terminal 200 and may input the manipulation instructionsto the cleaning robot 100 through the portable mobile terminal 200.

FIG. 2A is a block diagram of control flow of the cleaning robotillustrated in FIG. 1, FIG. 2B is a block diagram illustrating theconfiguration of a controller of the cleaning robot of FIG. 1, FIG. 3 isa perspective view illustrating the exterior of the cleaning robot ofFIG. 1, and FIG. 4 is a bottom view illustrating the bottom of thecleaning robot of FIG. 1.

Referring to FIGS. 2A through 4, the cleaning robot 100 includes amanipulation unit 110 that receives the manipulation instructions to thecleaning robot 100 from the user U, a display unit 115 that displaysinformation including operating information of the cleaning robot 100 tothe user U, an upward image capturing unit 120 that captures an image inan upward direction of the cleaning robot 100, an image capturing unit130 that captures an image viewed from the cleaning robot 100, a voiceinput unit 140 that receives a user's voice instructions, an obstacledetecting unit 150 that detects an obstacle in the space to be cleaned,a travelling unit 160 that moves the cleaning robot 100, a cleaning unit170 that cleans the space to be cleaned, a robot storing unit 180 thatstores programs and data for an operation of the cleaning robot 100, arobot communication unit 185 that communicates with the portable mobileterminal (see 200 of FIG. 1), and a robot controller 190 that controlsthe operation of the cleaning robot 100.

The manipulation unit 110 may include one or more manipulation buttons111 to which the manipulation instructions to the cleaning robot 100 areinput from the user U. The manipulation buttons 111 may include acleaning mode selection button to select a cleaning mode of the cleaningrobot 100, an operation/stop instruction button to instruct an operationand stop of the cleaning robot 100, and a charge return instructionbutton to instruct return to a charge station (not shown). In additionto the manipulation buttons 111, other types of a user interface such asa simple button, a membrane switch or a touch screen may be employed.

The display unit 115 includes a display panel 116 on which theinformation including the operating information of the cleaning robot100 is displayed to the user U. The operating information of thecleaning robot 100 including a current time, the state of a battery, anda cleaning mode may be displayed on the display panel 116. As thedisplay panel 116, a liquid crystal display (LCD) panel or a lightemitting diode (LED) panel may be employed.

The upward image capturing unit 120 may include a two-dimensional camera121 that is disposed on a top surface 101 of the cleaning robot 100 andcaptures an image in an upward direction of the cleaning robot 100,i.e., an image of a ceiling of the space to be cleaned. In addition, theupward image capturing unit 120 may include a graphic processor (notshown) that processes the image captured by the two-dimensional camera121 as needed. The graphic processor (not shown) may perform simpleimage processing like changing the size or resolution of the imagecaptured by the two-dimensional camera 121.

The image capturing unit 130 may include a three-dimensional camera 131that is disposed at the front portion of the cleaning robot 100 andcaptures a three-dimensional image viewed from the cleaning robot 100.The three-dimensional image includes information regarding a distancefrom a two-dimensional image of an object to be captured to the objectto be captured. As the three-dimensional camera 131, a stereo cameramodule or a depth sensor module may be employed.

The stereo camera module includes a plurality of two-dimensional camerasand determines distance information regarding the object to be capturedusing a difference between images captured by the plurality oftwo-dimensional cameras. Also, the stereo camera module outputsinformation regarding a distance between one of the images captured bythe plurality of two-dimensional cameras and the object to be captured.

The depth sensor module includes a two-dimensional camera that capturesan image of the object to be captured and an infrared sensor thatradiates infrared rays toward the object to be captured and detects thesize of the infrared rays reflected from the object to be captured,thereby measuring a distance to the object to be captured in atwo-dimensional image. Also, the depth sensor module outputs the imagecaptured by the two-dimensional camera and distance information obtainedby the infrared sensor.

The image capturing unit 130 may further include a graphic processorthat processes the three-dimensional image captured by thethree-dimensional camera 131 as needed.

The voice input unit 140 includes a plurality of microphones 141, 143,and 145 that obtain the user's voice and determine the direction of theuser U. In detail, the voice input unit 140 includes a front microphone141 disposed at the front portion of a top surface of the cleaning robot100, a left microphone 143 disposed at the left portion of the topsurface of the cleaning robot 100, and a right microphone 145 disposedat the right portion of the top surface of the cleaning robot 100. Thesemicrophones 141, 143, and 145 convert the user's voice signals intoelectrical signals and output the electrical signals. In this case, themicrophones 141, 143, and 145 may determine the direction of the user Uusing, for example, a difference in times at which the user's voicesignals are input to the three microphones 141, 143 and 145 andamplitudes of the voice signals.

The obstacle detecting unit 150 includes a plurality of infrared sensors151 that radiate infrared rays and measure the size of the infrared raysreflected from an obstacle, thereby determining the presence of theobstacle and a distance from the obstacle to the object to be captured.The plurality of infrared sensors 151 may be installed at the frontportion and the right portion and left portions of the cleaning robot100 at regular intervals, thereby determining the position of theobstacle. That is, when the infrared sensor 151 placed at the frontportion of the cleaning robot 100 detects the obstacle, it may bedetermined that the obstacle is present at the front portion of thecleaning robot 100, and when the infrared sensor 151 placed at the rightportion of the cleaning robot 100 detects the obstacle, it may bedetermined that the obstacle is present at the right portion of thecleaning robot 100.

The travelling unit 160 includes a plurality of travelling wheels 161and 163 that move the cleaning robot 100, and a roller 165 that assistswith the movement of the cleaning robot 100.

The plurality of travelling wheels 161 and 163 are disposed at the rightand left edge portions of the bottom surface of the cleaning robot 100so that the cleaning robot 100 may move forward or backward or mayrotate. For example, when the plurality of travelling wheels 161 and 163are rotated forward, the cleaning robot 100 moves forward, and when theplurality of travelling wheels 161 and 163 are rotated backward, thecleaning robot 100 moves backward. Also, when the plurality oftravelling wheels 161 and 163 are rotated in different directions, thecleaning robot 100 may rotate to the left or to the right on the spot.

The roller 165 is installed at the front portion of the bottom surfaceof the cleaning robot 100 and rotates according to a movement directionof the cleaning robot 100. Also, the roller 165 may allow the cleaningrobot 100 to be maintained in a stable position.

The cleaning unit 170 includes a main brush 171 that scatters dust inthe space to be cleaned, a plurality of side brushes 173 a and 173 bthat guide dust in the space to be cleaned toward the main brush 171,and a dust box 175 that inhales and stores dust scattered by the mainbrush 171.

The main brush 171 is disposed at a dust inhalation hole 103 formed inthe bottom surface of the cleaning robot 100. Also, the main brush 171scatters dust in the space to be cleaned into the dust inhalation hole103 while rotating about a rotation shaft parallel to the cleaning robot100.

The plurality of side brushes 173 a and 173 b are installed at the leftand right edge portions of the front of the bottom surface of thecleaning robot 100. That is, the side brushes 173 a and 173 b areinstalled approximately at a front of the plurality of travelling wheels161 and 163. The side brushes 173 a and 173 b sweep dust in a cleaningarea that may not be cleaned by the main brush 171 and guide dust towardthe main brush 171 while rotating about the rotation shaft perpendicularto the cleaning robot 100. Also, the side brushes 173 a and 173 b notonly may rotate on the spot but also are installed to protrude towardthe outside of the cleaning robot 100 so that an area to be cleaned bythe cleaning robot 100 may be enlarged.

The robot storing unit 180 may include a non-volatile memory (notshown), such as a magnetic disk or a solid state disk, in which acontrol program or control data to control an operation of the cleaningrobot 100 is permanently stored, and a volatile memory (not shown), suchas a dynamic random access memory (D-RAM) or a static random accessmemory (S-RAM), in which temporary data generated when the operation ofthe cleaning robot 100 is controlled is temporarily stored.

The robot communication unit 185 may include a wireless communicationmodule (not shown) that performs wireless communication with theportable mobile terminal (see 200 of FIG. 1) or the charge station (notshown) using a wireless communication method, such as wireless fidelity(W-Fi), Bluetooth, Zigbee, or near field communication (NFC).

The robot controller 190 includes a voice recognition module 191 thatdetects the user's manipulation instructions through the user's voicebased on the user's voice signals obtained by the voice input unit 140,a motion recognition module 193 that detects the user's manipulationinstructions according to the user's motion based on thethree-dimensional image captured by the image capturing unit 130, and amain control module 195 that controls the operation of the cleaningrobot 100 according to the user's manipulation instructions.

The voice recognition module 191 detects the user's manipulationinstructions by comparing voice instructions stored according to variousmanipulation instructions with the voice instructions input by the voiceinput unit 140. Here, the cleaning robot 100 may previously receive thevoice instructions corresponding to the manipulation instructions fromthe user U and may store the voice instructions, so as to improve aprobability that the voice recognition module 191 will recognize theuser's voice instructions.

The motion recognition module 193 detects positions of particularportions of the user U, such as a hand and a shoulder of the user U fromthe three-dimensional image and determines a trajectory of the handusing the detected positions of the hand and the shoulder. The motionrecognition module 193 detects the manipulation instructions intended bythe user U by comparing the determined trajectory of the hand withmotion instructions stored according to various manipulationinstructions. In addition, the motion recognition module 193 may detectthe position of the space to be cleaned instructed by the user's handusing the detected positions of the hand and the shoulder.

The main control module 195 controls the operation of the travellingunit 160 and the operation of the cleaning unit 170 based on the user'smanipulation instructions input through the manipulation unit 110, theimage capturing unit 130 and the voice input unit 140 and outputs of theupward image capturing unit 120 and the obstacle detecting unit 150. Forexample, when cleaning instructions are input through the manipulationunit 110, the controller 190 controls the travelling unit 160 so thatthe cleaning robot 100 may travel along a predetermined travelling path,and the controller 190 controls the cleaning unit 170 so that thecleaning robot 100 may remove dust on a travelling path along which thecleaning robot 100 travels.

The controller 190 may include a microprocessor that performs anarithmetic operation on the control program and data input according tothe control program stored in the robot storing unit 180 and outputs theresult of the arithmetic operation. Also, the controller 190 may includea general processor that performs all operations, or a plurality ofprocessors that perform a specialized function, such as graphicprocessors or communication processors.

FIG. 5 is a block diagram illustrating control flow of the portablemobile terminal illustrated in FIG. 1, and FIG. 6 is a perspective viewillustrating the exterior of the portable mobile terminal of FIG. 1.

Referring to FIGS. 5 and 6, the portable mobile terminal 200 includes auser interface 210 that receives the manipulation instructions from theuser U and displays a screen corresponding to the user's manipulationinstructions, a terminal storing unit 230 in which a program and data tocontrol the portable mobile terminal 200 are stored, a terminalcommunication unit 240 that communicates with the cleaning robot (see100 of FIG. 1), and a terminal controller 250 that controls an operationof the portable mobile terminal 200.

The user interface 210 includes a touch pad 211 a that receives theuser's manipulation instructions through the user's touch motion, adisplay panel 211 b on which information corresponding to themanipulation instructions input to the touch pad 211 a is displayed, apower button 213 that receives power on/off instructions of the portablemobile terminal 200, and a home button 215 that converts the screendisplayed on the display panel 211 b into a main screen. Here, as thedisplay panel 211 b, an LCD panel or an LED panel may be employed.

The touch pad 211 a and the display panel 211 b are formed integrallywith each other and constitute a touch screen panel (TSP) 211. In theTSP 211, the transparent touch pad 211 a may be installed on the displaypanel 211 b, a screen on which the manipulation instructions that may beselected by the user U are displayed may be displayed on the displaypanel 211 b and coordinates touched by the user U may be detectedthrough the touch pad 211 a. When the user U touches the TSP 211according to the screen displayed on the display panel 211 b, theportable mobile terminal 200 may recognize the manipulation instructionsintended by the user U by comparing the coordinates touched by the userU with coordinates in which the manipulation instructions are displayed.

In addition, the portable mobile terminal 200 may detect a user's touchmotion through the touch pad 211 a and may recognize the manipulationinstructions intended by the user according to the user's detected touchmotion.

The terminal storing unit 230 may include a non-volatile memory (notshown), such as a magnetic disk or a solid state disk, in which acontrol program or control data for controlling an operation of theportable mobile terminal 200 is permanently stored, and a volatilememory (not shown), such as a D-RAM or an S-RAM, in which temporary datagenerated when the operation of the portable mobile terminal 200 iscontrolled is temporarily stored.

The terminal communication unit 240 may include a wireless communicationmodule (not shown) that performs wireless communication with thecleaning robot (see 100 of FIG. 1) using a wireless communicationmethod, such as Wi-Fi, Bluetooth, Zigbee, or NFC.

The terminal controller 250 controls the TSP 211 so that the screendisplayed on the TSP 211 may be changed according to the manipulationinstructions input through the TSP 211 and the information input throughthe terminal communication unit 240. For example, when the image of thespace to be cleaned captured by the cleaning robot 100 is received fromthe terminal communication unit 240 and image display instructions ofthe space to be cleaned are input from the user U, the terminalcontroller 250 displays the received image of the space to be cleaned onthe TSP 211.

The configurations of the cleaning robot 100 and the portable mobileterminal 200 illustrated in FIG. 1 have been described above.Hereinafter, a method of setting a focused cleaning area and arestricted area using the cleaning robot 100 and the portable mobileterminal 200 of FIG. 1, in accordance with an embodiment of the presentdisclosure will be described.

FIG. 7 illustrates the case in which the cleaning robot of FIG. 1receives voice instructions from the user, and FIGS. 8A, 8B and 8Cillustrate the case in which the cleaning robot of FIG. 1 determines auser's position through the user's voice instructions.

As illustrated in FIGS. 7 and 8, when the user U inputs callinstructions to the cleaning robot 100 through voice while the cleaningrobot 100 performs cleaning in an auto-cleaning mode, the cleaning robot100 receives voice signals of the user U through the voice input unit(see 140 of FIG. 2A) disposed at the cleaning robot 100 and detects thecall instructions of the user U based on the received voice signals.Also, when the call instructions are detected, the cleaning robot 100rotates toward the user U.

In detail, the cleaning robot 100 may use the voice signals received byat least one of three microphones 141, 143, and 145 included in thevoice input unit (see 140 of FIG. 2A) so as to receive the voice signalsof the user U. For example, the cleaning robot 100 may use the voicesignals received by a front microphone 141 from among the frontmicrophone 141, a left microphone 143 and a right microphone 144 forvoice recognition.

Also, when the voice signals of the user U are received, the cleaningrobot 100 may analyze the voice signals input through the voicerecognition module (see 191 of FIG. 2B) and may detect the callinstructions of the user U according to the result of analysis. Forexample, the cleaning robot 100 compares the voice signals of the user Ureceived by the voice input unit (see 140 of FIG. 2A) with voice signalsaccording to a plurality of manipulation instructions, thereby detectingthat the voice signals of the user U are the call instructions.

Also, as a result of analyzing the voice signals of the user U, when itis determined that the voice signals of the user U are the callinstructions, the cleaning robot 100 determines a direction of the userU based on the voice signals input to the three microphones 141, 143 and145 and rotates toward the user U. In detail, the cleaning robot 100compares input times of the voice signals input to the three microphones141, 143 and 145 and the amplitudes of the voice signals, therebyestimating the direction of the user U.

For example, when a voice signal is first received by the frontmicrophone 141 and the amplitude of the voice signal input to the frontmicrophone 141 is the largest, the cleaning robot 100 may estimate thatthe user U is in an area A1 or A2 illustrated in FIG. 8B. Also, when avoice signal is received by the left microphone 143 before the rightmicrophone 145 receives the voice signal and the amplitude of the voicesignal input to the left microphone 143 is larger than the amplitude ofthe voice signal input to the right microphone 145, the cleaning robot100 may estimate that the user U is in the area A2 of FIG. 8B. Inaddition, the cleaning robot 100 may estimate the position of the user Ubased on a difference between a time at which the voice signal is inputto the left microphone 143 and a time at which the voice signal is inputto the right microphone 145 or a difference between the amplitude of thevoice signal input to the left microphone 143 and the amplitude of thevoice signal input to the right microphone 145.

As another example, when the voice signals of the user U are input inthe order of the left microphone 143, the front microphone 141 and theright microphone 145 and the amplitudes of the voice signals are in theorder of the left microphone 143, the front microphone 14 and the rightmicrophone 145, the cleaning robot 100 may determine that the user U isin an area B1 illustrated in FIG. 8B.

When the direction of the user U is determined, the cleaning robot 100rotates toward the user U, as illustrated in FIG. 8C. In this case, thecleaning robot 100 may rotate in a direction in which rotationaldisplacement is low.

FIG. 9 illustrates the case in which the cleaning robot of FIG. 1obtains an image of the user and the user's voice.

When the call instructions are input from the user U, the cleaning robot100 rotates toward the user U as illustrated in FIG. 9 and obtainstwo-dimensional images of the user U and the space to be cleaned anddistance information using the image capturing unit (see 130 of FIG. 2).

The user U may input cleaning area designating instructions to thecleaning robot 100 through motion or voice. For example, the user U maysay “Clean here” while instructing a focused cleaning area to beintensely cleaned or may say “Don't clean here” while instructing arestricted area in which the entry of the cleaning robot 100 isprohibited. In this way, when the user U inputs the cleaning areadesignating instructions to the cleaning robot 100 through motion orvoice, the cleaning robot 100 determines an area instructed by the userU through the motion of the user U and analyzes the voice of the user U,thereby determining manipulation instructions of the user U.

Determining of the manipulation instructions of the user U may beperformed by comparing the voice signal input through at least one of aplurality of microphones (see 141, 143 and 145 of FIG. 3) with the voicesignal according to a plurality of manipulation instructions and bydetecting the cleaning area designating instructions of the user U. Forexample, the cleaning robot 100 may determine whether the user Uinstructs the focused cleaning area or the restricted area.

When the cleaning area designating instructions are determined throughthe voice of the user U, the cleaning robot 10 analyzes the motion ofthe user U and determines an area instructed by the user U using thetwo-dimensional image and the distance information obtained by the imagecapturing unit (see 130 of FIG. 2A).

FIGS. 10A and 10B and FIGS. 11A and 11B illustrate the case in which thecleaning robot of FIG. 1 determines coordinates of an area instructed bythe user from the image of the user.

As illustrated in FIG. 10A, the cleaning robot 100 detects a face F ofthe user U from the images of the user U and the space to be cleaned.Detecting the face F may be performed using methods such as aknowledge-based method, whereby detection of components that constitutethe face F of a human being, such as symmetric eyes, a nose and a mouthunder the eyes, is attempted, an eigenface algorithm, whereby detectionof the face F is attempted by approximating an eigenvector regarding anauto-correlation matrix of an image of the face F, a support vectormachine (SVM) algorithm, whereby the face F is detected using the SVM,or a boosting approach algorithm, whereby a detection method having ahigh detection rate is constituted by linearly combining severaldetection methods having low detection rates. However, the presentdisclosure is not limited thereto. For example, any method to detect aface F of the user U from the images of the user U may be used.

The face F of the user U is detected so that the cleaning robot 100 maydetermine the accurate position of the user U and the image of the userU may be used as a criterion to detect particular portions of the userU, such as a hand and a shoulder. In addition, a distance between theuser U and the cleaning robot 100 may be determined from a distancebetween the face F of the user U and the cleaning robot 100.

As illustrated in FIG. 10B, after detecting the face F of the user U,the cleaning robot 100 detects the particular portions of the user U,such as a shoulder RS and a hand RH, from the image of the user U basedon the position of the face F of the user U.

As illustrated in FIG. 11A, when the shoulder RS and the hand RH of theuser U are detected from the three-dimensional image of the user U, thecleaning robot 100 may determine a distance d1 between the user U andthe cleaning robot 100, a distance d2 between the shoulder RS of theuser U and the cleaning robot 100, and a distance d3 between the hand RHof the user U and the cleaning robot 100 using the distance informationobtained by the image capturing unit (see 130 of FIG. 2A). Also, thecleaning robot 100 may determine a direction of the user U, a directionof the shoulder RS, and a direction of the hand RH from thetwo-dimensional image captured by the image capturing unit (see 130 ofFIG. 2A).

Also, the cleaning robot 100 may determine three-dimensional relativecoordinates of the shoulder RS and the hand RH of the user U based onthe distance d1 between the user U and the cleaning robot 100 and thedirection of the user U, the distance d2 between the shoulder RS of theuser U and the cleaning robot 100 and the direction of the shoulder RSand the cleaning robot 100 and the distance d3 between the hand RH ofthe user U and the cleaning robot 100 and the direction of the hand RH.Here, the three-dimensional relative coordinates of the shoulder RS andthe hand RH of the user U define coordinates in a three-dimensionalrelative coordinate system in which the position of the cleaning robot100 is set as an origin. The three-dimensional relative coordinatesystem defines a coordinate system in which the cleaning robot 100 isset as an origin, a front direction of the cleaning robot 100 from acleaning floor is set as a +y-axis, a right direction of the cleaningrobot 100 from the cleaning floor is set as an +x-axis and an upwarddirection of the cleaning robot 100 from the cleaning floor is set as a+z-axis.

When the three-dimensional coordinates of the shoulder RS and hand RH ofthe user U are determined, the cleaning robot 100 determines a directionvector SH toward the hand RH of the user U from the shoulder RS of theuser U and determines two-dimensional relative coordinates of theposition P instructed by the user U based on the direction vector SHtoward the hand RH of the user U from the shoulder RS of the user U andthe three-dimensional relative coordinates of the shoulder RS or thehand RH of the user U. In other words, since the position P instructedby the user U is on the cleaning floor, a position in which thedirection vector SH toward the hand RH of the user U from the shoulderRS of the user U and the floor of the space to be cleaned cross eachother will be the position P instructed by the user U and may berepresented as two-dimensional relative coordinates having only anx-axis coordinate and a y-axis coordinate.

Also, the cleaning robot 100 determines screen coordinates indicating towhich position of the area instructed by the user U the two-dimensionalimage captured by the image capturing unit corresponds (see 130 of FIG.2A). Since the three-dimensional camera (see 131 of FIG. 3) is fixed tothe cleaning robot 100, the two-dimensional relative coordinatesindicating the position of the area instructed by the user U on thecleaning floor and the screen coordinates indicating the position of thearea instructed by the user U in the two-dimensional image are inone-to-one correspondence, as illustrated in FIG. 11B. That is, thecleaning robot 100 may determine the screen coordinates in thetwo-dimensional image when the two-dimensional relative coordinates ofthe area instructed by the user U are obtained.

Also, the cleaning robot 100 determines absolute coordinates of the areainstructed by the user U using relative coordinates of the areainstructed by the user U and position information of the cleaning robot100 obtained by the image capturing unit (see 120 of FIG. 2A). Here, theabsolute coordinates are coordinates based on a predetermined criterionpoint, such as an entrance of the space to be cleaned or the chargestation (not shown) within the space to be cleaned and are differentfrom the relative coordinates based on the cleaning robot 100.

FIG. 12 illustrates the case in which the cleaning robot of FIG. 1transmits the image of the user and an image of the space to be cleanedand information regarding the area instructed by the user to theportable mobile terminal, and FIG. 13 illustrates the case in which thecleaning robot of FIG. 1 and the portable mobile terminal of FIG. 1display the image of the cleaning space and the information regardingthe area instructed by the user.

When position information of the area instructed by the user U withinthe two-dimensional image is determined, the cleaning robot 100 displaysthe two-dimensional image on the display panel (see 116 of FIG. 3) andoverlappingly displays the area instructed by the user U and the type ofthe area instructed by the user U on the two-dimensional image (whetherthe area instructed by the user U is the focused cleaning area or therestricted area). Also, the cleaning robot 100 transmits thetwo-dimensional image, the screen coordinates and the absolutecoordinates of the area instructed by the user U, the type of the areainstructed by the user U and position information of the cleaning robot100 in the space to be cleaned to the portable mobile terminal 200through the communication unit (see 185 of FIG. 2A).

When the two-dimensional image is received, the portable mobile terminal200 displays the received two-dimensional image on the TSP 211 andoverlappingly displays the area instructed by the user U on thetwo-dimensional image using the screen coordinates of the areainstructed by the user U. Here, overlapping display of the areainstructed by the user U on the two-dimensional image using the cleaningrobot 100 and the portable mobile terminal 200 may be performed usingaugmented reality. Augmented reality, a kind of virtual reality, is agraphic technique whereby a virtual object or virtual information issynthesized to an actual environment so that the virtual object orvirtual information appears as an object in an original environment.

As illustrated in FIG. 13, the cleaning robot 100 and the portablemobile terminal 200 display a cleaning area screen 300 on which thetwo-dimensional image and the area instructed by the user U, i.e., afirst focused cleaning area C1, are displayed. Here, the first focusedcleaning area C1 may be overlappingly displayed on the two-dimensionalimage based on the screen coordinates of the area instructed by the userU and the type of the area instructed by the user U.

The user U may check the position of the focused cleaning area C1instructed by the user U through the cleaning area screen 300 displayedon the display panel (see 116 of FIG. 12) of the cleaning robot 100 andthe TSP 211 of the portable mobile terminal 200.

In addition, the user U may additionally input the focused cleaning areaor the restricted area through the TSP (see 211 of FIG. 12) of theportable mobile terminal 200.

FIG. 14 illustrates the case in which information regarding the focusedcleaning area is input to the portable mobile terminal of FIG. 1, andFIGS. 15 and 16 illustrate the case in which information regarding therestricted area is input to the portable mobile terminal of FIG. 1.

As illustrated in FIG. 14, the user U may input focused cleaning areainput instructions through motion or voice, may touch a position to beadditionally designated as the focused cleaning area on the cleaningarea screen 300 displayed on the TSP (see 211 of FIG. 12) of theportable mobile terminal 200 or may touch and drag an area to bedesignated as the focused cleaning area, thereby designating a secondfocused cleaning area.

When the user U inputs the second focused cleaning area through the TSP(see 211 of FIG. 12), the portable mobile terminal 200 overlappinglydisplays a second focused cleaning area C2 on the two-dimensional image.That is, the portable mobile terminal 200 displays a cleaning areascreen (not shown) on which the first focused cleaning area C1 and thesecond focused cleaning area C2 overlap each other on thetwo-dimensional image of the space to be cleaned, through the TSP (see211 of FIG. 12).

When the user U wants to input the restricted area, the user U may inputrestricted area input instructions, may touch a position to bedesignated as the restricted area on the cleaning area screen 320displayed on the TSP (see 211 of FIG. 12) of the portable mobileterminal 200, as illustrated in FIGS. 15 and 16, or may touch and dragan area to be designated as the restricted area U1, thereby designatinga first restricted area W1 and a second restricted area W2.

When the user U inputs the first restricted area W1 and the secondrestricted area W2 through the TSP 211, the portable mobile terminal 200overlappingly displays the first restricted area W1 and the secondrestricted area W2 on the two-dimensional image. That is, the portablemobile terminal 200 displays the cleaning area screen 330 on which thefirst focused cleaning area C1, the second focused cleaning area C2, thefirst restricted area W1 and the second restricted area W2 are displayedon the two-dimensional image of the space to be cleaned using augmentedreality, through the TSP (see 211 of FIG. 12).

FIG. 17 illustrates a screen on which the portable mobile terminal ofFIG. 1 displays the focused cleaning area and the restricted area.

The portable mobile terminal 200 may display positions of the focusedcleaning areas C1 and C2 and the restricted areas W1 and W2 in a planview (map) of the space to be cleaned.

In detail, when the user U completes input of the focused cleaning areaand the restricted area, the portable mobile terminal 200 determinestwo-dimensional relative coordinates of the second focused cleaning areaC2, the first restricted area W1 and the second restricted area W2 basedon the screen coordinates of the second focused cleaning area C2, thefirst restricted area W1 and the second restricted area W2. Thereafter,the portable mobile terminal 100 determines absolute coordinates of thesecond focused cleaning area C2, the first restricted area W1 and thesecond restricted area W2, i.e., positions of the second focusedcleaning area C2, the first restricted area W1 and the second restrictedarea W2 within the space to be cleaned using the position of thecleaning robot 100 when the cleaning robot 100 captures thetwo-dimensional image and the two-dimensional relative coordinates ofthe second focused cleaning area C2, the first restricted area W1 andthe second restricted area W2. Also, the portable mobile terminal 200transmits the absolute coordinates of the second focused cleaning areaC2, the first restricted area W1 and the second restricted area W2 tothe cleaning robot 100 so that the cleaning robot 100 may detect thefocused cleaning areas C1 and C2 and the restricted areas W1 and W2.

When the absolute coordinates of the focused cleaning areas C1 and C2and the restricted areas W1 and W2 are determined, the portable mobileterminal 200 may display a map of the space to be cleaned and maydisplay positions of the focused cleaning areas C1 and C2 and therestricted areas W1 and W2 on the map of the space to be cleaned.

FIG. 18 is a flowchart illustrating a method of inputting and displayingthe focused cleaning area and the restricted area using the cleaningrobot and the portable mobile terminal in accordance with an embodimentof the present disclosure.

Simply describing the method of inputting and displaying the focusedcleaning area and the restricted area using the cleaning robot (see 100of FIG. 1) and the portable mobile terminal (see 200 of FIG. 1) inaccordance with an embodiment of the present disclosure with referenceto FIG. 18, it is determined whether the user's manipulationinstructions are input to the cleaning robot (see 100 of FIG. 1) throughvoice while the space to be cleaned is cleaned (Operation 505).

When the manipulation instructions are input through voice (“YES” inOperation 505), the cleaning robot (see 100 of FIG. 1) detects themanipulation instructions (Operation 508). In detail, as a result ofdetecting the manipulation instructions, when the manipulationinstructions are call instructions, the cleaning robot (see 100 ofFIG. 1) determines a position of the user and rotates toward theposition of the user.

Next, the cleaning robot (see 100 of FIG. 1) captures an image of theuser (Operation 510).

Next, the cleaning robot (see 100 of FIG. 1) detects a motion of theuser (Operation 515). In detail, the cleaning robot (see 100 of FIG. 1)detects the user's focused cleaning area input instructions orrestricted area input instructions through the image of the user.

Next, the cleaning robot (see 100 of FIG. 1) determines a position ofthe focused cleaning area or the restricted area through the user'smotion (Operation 520).

Next, the cleaning robot (see 100 of FIG. 1) displays the focusedcleaning area or the restricted area (Operation 525). Also, the cleaningrobot (see 100 of FIG. 1) may transmit information regarding the focusedcleaning area or the restricted area to the portable mobile terminal(see 200 of FIG. 1), and the portable mobile terminal (see 200 ofFIG. 1) may display the focused cleaning area or the restricted area.

Next, the portable mobile terminal (see 200 of FIG. 1) determineswhether the user additionally inputs the focused cleaning area or therestricted area (Operation 530).

When the user additionally inputs the focused cleaning area or therestricted area (“YES” in Operation 530), the portable mobile terminal(see 200 of FIG. 1) determines a position of the focused cleaning areaor the restricted area (Operation 535) and displays the focused cleaningarea or the restricted area (Operation 540).

Also, the portable mobile terminal (see 200 of FIG. 1) displays thefocused cleaning area or the restricted area on a map of a space to becleaned on which the space to be cleaned is displayed (Operation 545).

When it is determined that the user does not additionally input thefocused cleaning area or the restricted area (“NO” in Operation 530),the portable mobile terminal (see 200 of FIG. 1) displays the focusedcleaning area or the restricted area transmitted by the cleaning robot(see 100 of FIG. 1) on the map of the space to be cleaned on which thespace to be cleaned is displayed.

The method of setting the focused cleaning area and the restricted areausing the cleaning robot and the portable mobile terminal illustrated inFIG. 1 in accordance with an embodiment of the present disclosure hasbeen described above. Hereinafter, a method of setting the focusedcleaning area and the restricted area using the cleaning robot and theportable mobile terminal illustrated in FIG. 1 in accordance withanother embodiment of the present disclosure will be described.

FIG. 19 illustrates the case in which the cleaning robot of FIG. 1receives manipulation instructions from the user through the portablemobile terminal.

Referring to FIG. 19, the cleaning robot 100 receives the user'smanipulation instructions through the portable mobile terminal 200during a manual cleaning mode.

In detail, the cleaning robot 100 transmits an image captured by theimage capturing unit (see 130 of FIG. 2A) to the portable mobileterminal 200, and the user U may manipulate a travelling direction ofthe cleaning robot 100 using a direction key 221 b disposed on the TSP211 of the portable mobile terminal 200 while watching an image 221 a ofthe cleaning robot 100 displayed on the TSP 211 of the portable mobileterminal 200.

The user U may input focused cleaning area input instructions orrestricted area input instructions through a motion instruction inputarea 221 c disposed on the TSP 211 of the portable mobile terminal 200during the travelling manipulation of the cleaning robot 100.

When the focused cleaning area input instructions or the restricted areainput instructions are input from the user, the portable mobile terminal200 displays a two-dimensional image captured by the cleaning robot 100when the focused cleaning area input instructions or the restricted areainput instructions are input and informs the cleaning robot 100 that thefocused cleaning area input instructions or the restricted area inputinstructions are input. The cleaning robot 100 that has been informedthat the focused cleaning area input instructions or the restricted areainput instructions are input transmits position information of thecleaning robot 100 within the space to be cleaned to the portable mobileterminal 200.

FIG. 20 illustrates a screen on which information regarding the focusedcleaning area is input through the portable mobile terminal of FIG. 1,and FIGS. 21 and 22 illustrate a screen on which information regardingthe restricted area is input through the portable mobile terminal ofFIG. 1.

The user U may input focused cleaning area input instructions and maytouch a position to be designated as the focused cleaning area on acleaning area screen 400 displayed on the TSP (see 211 of FIG. 19) ofthe portable mobile terminal 200, as illustrated in FIG. 20 or may touchand drag an area to be designated as the focused cleaning area, therebydesignating a first focused cleaning area C1.

When the user U inputs the first focused cleaning area C1 through theTSP 211, the portable mobile terminal 200 displays a cleaning areascreen (not shown) on which the first focused cleaning area C1 isdisplayed on the two-dimensional image using augmented reality.

Also, when the user U wants to additionally input a second focusedcleaning area C2 in addition to the first focused cleaning area C1, theuser U may touch a position to be designated as the focused cleaningarea on the TSP (see 211 of FIG. 19) of the portable mobile terminal 200or may touch and drag an area to be designated as the focused cleaningarea, thereby designating the second focused cleaning area C2. Theportable mobile terminal 200 displays a cleaning area screen 410 onwhich the first focused cleaning area C1 and the second focused cleaningarea C2 are displayed on the two-dimensional image through the TSP (see211 of FIG. 19) using augmented reality.

When the user U wants to input the restricted area, the user U may inputrestricted area input instructions and may touch a position to bedesignated as the restricted area on the TSP (see 211 of FIG. 19) of theportable mobile terminal 200, as illustrated in FIGS. 21 and 22 or maytouch and drag an area to be designated as the focused cleaning area,thereby designating a first restricted area W1 and a second restrictedarea W2.

When the user U inputs the first restricted area W1 and the secondrestricted area W2 through the TSP (see 211 of FIG. 19), the portablemobile terminal 200 displays a cleaning area screen 420 on which thefirst restricted area W1 and the second restricted area W2 are displayedon the two-dimensional image using augmented reality.

FIG. 23 illustrates a screen on which the portable mobile terminal ofFIG. 1 displays the focused cleaning area and the restricted area.

The portable mobile terminal 200 may display positions of the focusedcleaning areas C1 and C2 and the restricted areas W1 and W2 in a planview (map) of a space to be cleaned, as illustrated in FIG. 23.

When the user U completes input of the focused cleaning areas C1 and C2and the restricted areas W1 and W2, two-dimensional relative coordinatesof the focused cleaning areas C1 and C2 and the restricted areas W1 andW2 are determines based on screen coordinates of the focused cleaningareas C1 and C2 and the restricted areas W1 and W2. Thereafter, theportable mobile terminal 200 determines absolute coordinates of thefocused cleaning areas C1 and C2 and the restricted areas W1 and W2,i.e., positions of the focused cleaning areas C1 and C2 and therestricted areas W1 and W2 within the space to be cleaned, using aposition of the cleaning robot 100 when the cleaning robot 100 capturesa two-dimensional image and two-dimensional relative coordinates of thefocused cleaning areas C1 and C2 and the restricted areas W1 and W2.Also, the portable mobile terminal 200 transmits the absolutecoordinates of the focused cleaning areas C1 and C2 and the restrictedareas W1 and W2 to the cleaning robot 100 so that the cleaning robot 100may detect the focused cleaning areas C1 and C2 and the restricted areasW1 and W2.

When the absolute coordinates of the focused cleaning areas C1 and C2and the restricted areas W1 and W2 are determined, the portable mobileterminal 200 may display the map of the space to be cleaned and maydisplay positions of the focused cleaning areas C1 and C2 and therestricted areas W1 and W2 on the map of the space to be cleaned.

FIG. 24 is a flowchart illustrating a method of inputting and displayingthe focused cleaning area and the restricted area using the cleaningrobot and the portable mobile terminal of FIG. 1 in accordance withanother embodiment of the present disclosure.

Simply describing the method of inputting and displaying the focusedcleaning area and the restricted area using the cleaning robot (see 100of FIG. 1) and the portable mobile terminal (see 200 of FIG. 1) inaccordance with another embodiment of the present disclosure withreference to FIG. 18, it is determined whether the user's cleaning areainput instructions or restricted input instructions are input to thecleaning robot (see 100 of FIG. 1) through the portable mobile terminal(see 200 of FIG. 1) while the space to be cleaned is cleaned in a manualmode (Operation 605).

When the user's cleaning area input instructions or restricted inputinstructions are input through the portable mobile terminal (see 200 ofFIG. 1; “YES” in Operation 605), the cleaning robot (see 100 of FIG. 1)captures an image of a space to be cleaned (Operation 610). Also, thecleaning robot (see 100 of FIG. 1) transmits the captured image of thespace to be cleaned to the portable mobile terminal (see 200 of FIG. 1).

Next, the portable mobile terminal (see 200 of FIG. 1) displays theimage of the space to be cleaned (Operation 615).

Next, the portable mobile terminal (see 200 of FIG. 1) determineswhether the user inputs a focused cleaning area or a restricted area(Operation 620).

When it is determined that the user inputs the focused cleaning area orthe restricted area (“YES” in Operation 620), the portable mobileterminal (see 200 of FIG. 1) determines a position of the focusedcleaning area or the restricted area (Operation 625) and displays thefocused cleaning area or the restricted area (Operation 630).

Also, the portable mobile terminal (see 200 of FIG. 1) displays thefocused cleaning area or the restricted area on a map of a space to becleaned on which the space to be cleaned is displayed (Operation 635).

When it is determined that the user does not input the focused cleaningarea or the restricted area (“NO” in Operation 620), the portable mobileterminal (see 200 of FIG. 1) displays the map of the space to be cleanedon which the space to be cleaned is displayed (Operation 635).

Although a few embodiments of the present disclosure have been shown anddescribed, it would be appreciated by those skilled in the art thatchanges may be made in these embodiments without departing from theprinciples and spirit of the disclosure, the scope of which is definedin the claims and their equivalents.

What is claimed is:
 1. An auto-cleaning system comprising: a cleaningrobot configured to perform cleaning while travelling a space to becleaned; and a user terminal configured to control the cleaning robotremotely, wherein the cleaning robot receives a voice input of a user,and in response to the receiving of the voice input of the user, thecleaning robot captures an image of the space to be cleaned, identifiesa position indicated by the user based on the captured image of thespace to be cleaned, and transmits, to the user terminal, the capturedimage of the space to be cleaned and information regarding the positionindicated by the user, and the user terminal receives, from the cleaningrobot, the image of the space to be cleaned and the informationregarding the position indicated by the user, displays the image of thespace to be cleaned and the information regarding the position indicatedby the user, and receives information regarding another position fromthe user.
 2. The auto-cleaning system of claim 1, wherein the cleaningrobot comprises an image capturing unit configured to capture the imageof the space to be cleaned and receive a motion of the user, and a voiceinput unit configured to receive a voice of the user, and the cleaningrobot identifies a focused cleaning area and/or a restricted area basedon the voice input through the voice input unit and the motion inputthrough the image capturing unit.
 3. The auto-cleaning system of claim2, wherein, when the voice is input through the voice input unit, thecleaning robot rotates toward the user and receives the motion throughthe image capturing unit.
 4. The auto-cleaning system of claim 1,wherein the user terminal comprises a touch screen panel on which theimage of the space to be cleaned is displayed and a user input isreceived from the user, and when the information regarding the anotherposition from the user is input through the touch screen panel, theinformation regarding the another position from the user isoverlappingly displayed on the image of the space to be cleaned throughthe touch screen panel.
 5. The auto-cleaning system of claim 4, whereinthe user terminal determines absolute coordinates in the space to becleaned that correspond to the another position from the user.
 6. Theauto-cleaning system of claim 5, wherein the user terminal identifiesimage coordinates of the another position from the user, identifiesrelative coordinates of the another position from the user in which thecleaning robot is set as an origin, based on the image coordinates, andidentifies absolute coordinates of the another position from the user inthe space to be cleaned, based on the relative coordinates and aposition of the cleaning robot.
 7. An auto-cleaning method comprising:moving a cleaning robot in a space to be cleaned; receiving a voiceinput of a user; in response to the receiving of the voice input of theuser, capturing, by the cleaning robot, an image of the space to becleaned; identifying, by the cleaning robot, a position indicated by theuser based on the captured image of the space to be cleaned;transmitting, from the cleaning robot, the image of the space to becleaned and information regarding the position indicated by the user toa user terminal; displaying, on the user terminal, the image of thespace to be cleaned and the information regarding the position indicatedby the user received from the cleaning robot; and receiving, by the userterminal, information regarding another position from the user.
 8. Theauto-cleaning method of claim 7, wherein the identifying of the positionindicated by the user includes: identifying a focused cleaning areaand/or a restricted area based on a voice of the user input through avoice input unit and the motion of the user input through an imagecapturing unit.
 9. The auto-cleaning method of claim 8, wherein theidentifying of the position indicated by the user further includes:rotating the cleaning robot toward the user when the voice of the useris input through the voice input unit; and capturing the motion of theuser through the image capturing unit.
 10. The auto-cleaning method ofclam 7, wherein the information regarding the another position from theuser is input through a touch screen panel of the user terminal, and theinformation regarding the another position from the user isoverlappingly displayed on the image of the space to be cleaned throughthe touch screen panel.
 11. The auto-cleaning method of claim 10,further comprising determining, by the user terminal, absolutecoordinates in the space to be cleaned that correspond to the anotherposition from the user.
 12. The auto-cleaning method of claim 11,wherein the determining of the absolute coordinates includes:identifying image coordinates of the another position from the user fromthe touch screen panel; identifying relative coordinates of the anotherposition from the user in which the cleaning robot is set as an origin,based on the image coordinates; and identifying absolute coordinates ofthe another position from the user in the space to be cleaned, based onthe relative coordinates and a position of the cleaning robot.